68

6

The Nature of Information

ple, a language with the predicates W, M, and C, designating respectively warm,

moderate, and cold temperatures, would be efficient in a continental climate (e.g.,

Switzerland or Hungary) but would become inefficient with a move to the western

margin of Europe, since M occurs much more frequently there.

Although the quantification of information is deliberately abstracted from the

content of a message, taking content into account may allow much more dramatic

compression of a message than is possible using solely the statistical redundancy

(Eq. 6.18). Consider how words such as “utilization” may be replaced by “use”,

appellations such as “guidance counsellor” by “counsellor”, and phrases such as “at

this moment in time” by “at this moment”, or simply “now”. Many documents can be

thus reduced in length by over two-thirds without any loss in meaning (but a consid-

erable gain in readability). With simply constructed texts, algorithmic procedures for

accomplishing this that do not require the text to be interpreted can be devised; for

example, all the words in the text can be counted and listed in order of frequency of

occurrence, and then each sentence is assigned a score according to the numbers of

the highest-ranking words (apart from “and”, “that”, etc.) it contains. The sentences

with the highest scores are preferentially retained. ²⁸

6.3.3

Effect

A signal may be accurately received and its meaning may be understood by the

recipient, but that does not guarantee that it will engender the response desired by

the sender. This aspect of information deals with the ultimate result and the possibly

far-reaching consequences of a message and how the deduced meaning is related to

human purposes. The question of the value of information has already been discussed

(Sect. 6.2.1), and operationally it comes close to a quantification of effect.

Mackay has proposed that the quantum of effective information is that amount

that enables the recipient to make one alteration to the logical pattern describing his

awareness of the relevant situation, and this would appear to provide a good basis for

quantifying effect. Suppose that an agent has a state of mind upper M 1M1, which comprises

certain beliefs, hypotheses, and the like (the prior state). The agent then hears a

sentence, which causes a change to state of mind upper M 2M2, the posterior state, which

stands in readiness to make a response. If the meaning of an item of information is

its contribution to the agent’s total state of conditional readiness for action and the

planning of action (i.e., the agent’s conditional repertoire of action), then the effect

is the ultimate realization of that conditional readiness in terms of actual action. ²⁹

As soon as we introduce the notion of a conditional repertoire of action, we see

that selection must be considered. Indeed, the three essential attributes of an agent

are (and note the parallel with the symbolic level) as follows:

28 It should be noted that this strategy is easily defeated by increasing the complexity of sentence

construction.

29 Wiener subsumes effect into meaning in his definition of “meaningful information.”